Application of the Rasch measurement framework to mammography positioning data.

The purpose of this article is to provide raw data and measure-validation data pertaining to a co-submission published in European Journal of Radiology and entitled: Development and validation of a novel measure of adverse patient positioning in mammography. This Data in Brief article serves not only to provide greater detail than its companion article but also as an educational worked example of the Rasch measurement framework. Rasch measurement is a form of modern psychometric technique and our articles provide the first known example of its use in the evaluation of clinical radiological image quality. The data consist of observations of mammographic images, plus limited participant parameters relevant to the measure validation process. Also provided are validation indices produced by subjecting the primary data to Rasch analysis. An expert observer generated the primary data by reviewing mammographic images to judge the presence or absence of a set of features developed through theory and consultation with other experts. The validation data were generated through Rasch analysis, performed using Winsteps® software, which mathematically models the probability of having a correct response (or a present feature in this dataset) to an item in a given measurement instrument (e.g. questionnaire), as a function of the participant's ability/position on the underlying construct under study. The data can be reused by anyone wishing to learn and practice psychometric validation techniques. They can also form a basis for researchers wishing to build on our preliminary measure for the assessment of mammographic clinical image quality.

Hospital length of stay for COVID-19 patients: Data-driven methods for forward planning.

BACKGROUND: Predicting hospital length of stay (LoS) for patients with COVID-19 infection is essential to ensure that adequate bed capacity can be provided without unnecessarily restricting care for patients with other conditions. Here, we demonstrate the utility of three complementary methods for predicting LoS using UK national- and hospital-level data. METHOD: On a national scale, relevant patients were identified from the COVID-19 Hospitalisation in England Surveillance System (CHESS) reports. An Accelerated Failure Time (AFT) survival model and a truncation corrected method (TC), both with underlying Weibull distributions, were fitted to the data to estimate LoS from hospital admission date to an outcome (death or discharge) and from hospital admission date to Intensive Care Unit (ICU) admission date. In a second approach we fit a multi-state (MS) survival model to data directly from the Manchester University NHS Foundation Trust (MFT). We develop a planning tool that uses LoS estimates from these models to predict bed occupancy. RESULTS: All methods produced similar overall estimates of LoS for overall hospital stay, given a patient is not admitted to ICU (8.4, 9.1 and 8.0 days for AFT, TC and MS, respectively). Estimates differ more significantly between the local and national level when considering ICU. National estimates for ICU LoS from AFT and TC were 12.4 and 13.4 days, whereas in local data the MS method produced estimates of 18.9 days. CONCLUSIONS: Given the complexity and partiality of different data sources and the rapidly evolving nature of the COVID-19 pandemic, it is most appropriate to use multiple analysis methods on multiple datasets. The AFT method accounts for censored cases, but does not allow for simultaneous consideration of different outcomes. The TC method does not include censored cases, instead correcting for truncation in the data, but does consider these different outcomes. The MS method can model complex pathways to different outcomes whilst accounting for censoring, but cannot handle non-random case missingness. Overall, we conclude that data-driven modelling approaches of LoS using these methods is useful in epidemic planning and management, and should be considered for widespread adoption throughout healthcare systems internationally where similar data resources exist.

Length of Stay in ICU of Covid-19 patients in England, March - May 2020.

INTRODUCTION: Length of Stay (LoS) in Intensive Care Units (ICUs) is an important measure for planning beds capacity during the Covid-19 pandemic. However, as the pandemic progresses and we learn more about the disease, treatment and subsequent LoS in ICU may change. OBJECTIVES: To investigate the LoS in ICUs in England associated with Covid-19, correcting for censoring, and to evaluate the effect of known predictors of Covid-19 outcomes on ICU LoS. DATA SOURCES: We used retrospective data on Covid-19 patients, admitted to ICU between 6 March and 24 May, from the "Covid-19 Hospitalisation in England Surveillance System" (CHESS) database, collected daily from England's National Health Service, and collated by Public Health England. METHODS: We used Accelerated Failure Time survival models with Weibull and log-normal distributional assumptions to investigate the effect of predictors, which are known to be associated with poor Covid-19 outcomes, on the LoS in ICU. RESULTS: Patients admitted before 25 March had significantly longer LoS in ICU (mean = 18.4 days, median = 12), controlling for age, sex, whether the patient received Extracorporeal Membrane Oxygenation, and a co-morbid risk factors score, compared with the period after 7 April (mean = 15.4, median = 10). The periods of admission reflected the changes in the ICU admission policy in England. Patients aged 50-65 had the longest LoS, while higher co-morbid risk factors score led to shorter LoS. Sex and ethnicity were not associated with ICU LoS. CONCLUSIONS: The skew of the predicted LoS suggests that a mean LoS, as compared with median, might be better suited as a measure used to assess and plan ICU beds capacity. This is important for the ongoing second and any future waves of Covid-19 cases and potential pressure on the ICU resources. Also, changes in the ICU admission policy are likely to be confounded with improvements in clinical knowledge of Covid-19.

Development and validation of a novel measure of adverse patient positioning in mammography.

PURPOSE: The primary aim was to develop and validate a novel mammography positioning measure, specifically incorporating parameters which might relate to mammography pain. We then explored relationships between the new adverse positioning score and (1) pain; (2) patient and technique factors. METHODS: A 15-item instrument incorporating positioning features with potential to relate to mammography pain was developed. Participants' mammograms (n = 310) were reviewed for presence of these features. Validity was investigated using the Rasch model. Scores produced by the resultant measure were investigated for associations with patients' pain scores and relevant patient and technique factors, using Pearson correlation, analysis of variance, and multiple linear regression. RESULTS: Statistical indices within the Rasch measurement framework provided good evidence that the measure reflected a coherent construct of adverse positioning. Thus, the scores produced with the measurement instrument were valid for use in further statistical analysis. There is, however, scope for improvement of the measure's discriminatory properties. Adverse positioning scores were higher for greater breast volumes (r = 0.12, p=.0391) and body mass index (BMI) (r = 0.13, p=.0349), and varied by mammographer (F(11,298) 2.38, p = .0078). The relationships with BMI and mammographer persisted in regression modelling. No relationship was found between adverse positioning and pain. CONCLUSIONS: Evidence from Rasch analysis suggests that this novel measure is valid for quantifying a coherent "adverse positioning" construct in mammography. Adverse positioning scores varied by mammographer and were related to higher patient BMI but not to mammography pain. The measure warrants expansion, further refinement, and testing in larger studies.

Validating a Measure of Numeracy Skill Use in the Workplace for Incarcerated and Household Adults.

The aim of this study is to construct a measure of numeracy skill use in the workplace for incarcerated and household adults. The 2012/2014 Programme for the International Assessment of Adult Competencies (PIAAC) Survey of Adult Skills asked about the type and frequency of numeracy tasks performed as part of one's job to nationally-representative incarcerated and household adult samples. This paper takes these items from this survey and focuses on the validation of a measure of numeracy skill use in the workplace using the principles of the Rasch rating scale model. In the interest of exploring options for strengthened validity, response categories were collapsed to produce an optimal categorization structure. Findings suggest an instrument to measure numeracy skill use in prison and free market workplaces could potentially be improved with fewer response categories and more items that ask about a broader range of numeracy skills.

Developing a short-form of the Genetic Counselling Outcome Scale: The Genomics Outcome Scale.

The Genetic Counselling Outcome Scale (GCOS-24) is a 24-item patient reported outcome measure for use in evaluations of genetic counselling and testing services. The aim of this study was to develop a short form of GCOS-24. The study comprised three phases. Phase I: Cognitive interviews were used to explore interpretability of GCOS-24 items and which GCOS-24 items were most valued by the target population. Phase II: The Graded Response Model was used to analyse an existing set of GCOS-24 responses (n = 395) to examine item discrimination. Phase III: Item Selection. Three principles guided the approach to item selection (i) Items with poor discriminative properties were not selected; (ii) To avoid redundancy, items capturing a similar outcome were not selected together; item information curves and cognitive interview findings were used to establish superior items. (iii) Rasch analysis was then used to determine the optimal scale. In Phase I, ten cognitive interviews were conducted with individuals affected by or at risk for a genetic condition, recruited from patient support groups. Analysis of interview transcripts identified twelve GCOS-24 items which were highly valued by participants. In Phase II, Graded Response Model item characteristic curves and item information curves were produced. In Phase III, findings from Phases I and II were used to select ten highly-valued items that perform well. Finally, items were iteratively removed and permutated to establish optimal fit statistics under the Rasch model. A six-item questionnaire with a five-point Likert Scale was produced (The Genomics Outcome Scale (GOS)). Correlation between GCOS-24 and GOS scores is high (r = 0.838 at 99% confidence), suggesting that GOS maintains the ability of GCOS-24 to capture empowerment, whilst providing a less burdensome scale for respondents. This study represents the first step in developing a preference-based measure which could be used in the evaluation of technologies and services used in genomic medicine.

Rasch analysis of the Antimicrobial Self-Assessment Toolkit for National Health Service (NHS) Trusts (ASAT v17).

OBJECTIVES: The Antimicrobial Self-Assessment Toolkit for National Health Service (NHS) Trusts (ASAT) was developed to evaluate hospital-based antimicrobial stewardship programmes. Iterative validity investigations of the ASAT were used to produce a 91-item ASAT v17 utilizing qualitative methodology. Rasch analysis was used to generate question (item) behaviour estimates and to investigate the validity of ASAT v17. METHODS: In 2012, the partial credit model (PCM) was used to analyse ASAT responses from 33 NHS Trusts within England. WINSTEPS® outputs such as fit statistics and respondent/item maps were examined to determine unidimensionality, item discrimination and item hierarchy. Ordinary least squares regression modelling was used to determine the predictive validity using NHS Trust ability estimates generated from the PCM and corresponding Clostridium difficile rates. RESULTS: Each domain contained items that were misfitting the PCM (with INFIT MNSQ <0.7 or >1.3), except Domain 3. Subsequent iterative item removal had a negligible effect on the fit indices within most ASAT domains. Scale analysis demonstrated that most items were productive for measurement (n = 81). Respondent/item maps showed ceiling effects (n = 3) and floor effects (n = 1) within ASAT domains. Ordinary least squares regression modelling identified that there was limited predictive validity due to the small positive correlation between the predictor and outcome variables for participating hospitals (ρ = 0.146; P = 0.418). CONCLUSIONS: Rasch analysis was an effective measurement technique for evaluating the validity of ASAT v17 by providing evidence that each sub-scale and the overall scale demonstrated unidimensionality (construct validity). Improved item targeting may be required to improve item discrimination within the toolkit.

Measuring alternative learning outcomes: dispositions to study in higher education.

In this paper we describe the validation of two scales constructed to measure pre-university students' changing disposition (i) to enter Higher Education (HE) and (ii) to further study mathematically-demanding subjects. Items were selected drawing on interview data, and on a model of disposition as socially- as well as self- attributed. Rasch analyses showed that the two scales each produce robust one-dimensional measures on what we call a 'strength of commitment to enter HE' and 'disposition to study mathematically-demanding subjects further' respectively. However, the former scale was initially found to suffer psychometrically from a ceiling effect, which we 'corrected' by adding some harder items at a later data point, and revised the scale according to our interpretation of subsequent results. We finally discuss the potential significance of the constructed measures of learning outcomes, as variables in monitoring or even explaining students' progress into different subjects in HE.